Advanced Free Viewpoint Video Streaming Techniques

Free-viewpoint video is a new type of interactive multimedia service allowing users to control their viewpoint and generate new views of a dynamic scene from any perspective. The uniquely generated and displayed views are composed from two or more high bitrate camera streams that must be delivered to the users depending on their continuously changing perspective. Due to significant network and computational resource requirements, we proposed scalable viewpoint generation and delivery schemes based on multicast forwarding and distributed approach. Our aim was to find the optimal deployment locations of the distributed viewpoint synthesis processes in the network topology by allowing network nodes to act as proxy servers with caching and viewpoint synthesis functionalities. Moreover, a predictive multicast group management scheme was introduced in order to provide all camera views that may be requested in the near future and prevent the viewpoint synthesizer algorithm from remaining without camera streams. The obtained results showed that even 42% traffic decrease can be realized using distributed viewpoint synthesis and the probability of viewpoint synthesis starvation can be also significantly reduced in future free viewpoint video services

Free Viewpoint Television: új perspektíva a 3D videó továbbításban

A mélységérzetet visszaadó képmegjelenítés hosszú múltra tekint vissza, hiszen az első 3D technikát, a színszűréses, más néven anaglif szemüveges módszert már több mint 160 éve, 1853-ban dolgozta ki Wilhelm Rollmann. Azóta számos eljárás és technológia látott napvilágot, melyek népszerűsége erősen ingadozott az évtizedek folyamán. Az elmúlt néhány évben az otthoni 3D TV-k elterjedésének lehettünk tanúi, azonban a kezdeti, erős marketinggel fűtött lelkesedés mára alábbhagyott. Igazán nagy áttörés tehát most sem született. A színszűrős módszert felváltotta a polárszűrős és aktívzáras megoldás, azonban a kényelmetlenséget jelentő szemüveg használata továbbra is nélkülözhetetlen. Az igazi, maradandó áttörés a 3D televíziózás területén akkor várható, ha sikerül szemüveg nélkül is mélységérzetet visszaadó képmegjelenítést megvalósítani. Hasonló folyamatokat láthattunk a mozi esetén is, azonban ott a szemüveg okozta kényelmetlenséget jobban tolerálják a nézők. A 3D megjelenítés tökéletesítése folyamatos motivációt ad a kutatóknak és a fejlesztőmérnököknek. Ma már megjelentek olyan szemüveg nélkül is nézhető 3D kijelzők prototípusai, mely reményeink szerint már valóban meghozzák az áttörést. Az új technológia a sokkamerás felvételeken és a tetszés szerint nézőpont előállításon alapul

Free Viewpoint Video Streaming: Concepts, Techniques and Challenges

In contrast to traditional 3D videos, which offer the users only a single viewpoint, Free-Viewpoint Video (FVV) is a promising approach to allow free perspective selection while watching multi-view video streams. The user-specific views dynamically change based on the user’s position, and they must be synthetized accordingly. The unique views are synthesized from two or more high bitrate camera streams and the corresponding depth maps that must be delivered over the network and displayed with low latency. By increasing the number of deployed cameras and the density of the camera setup, the free-viewpoint video experience becomes more realistic. But on the other hand, more camera streams require higher network capacity. Therefore, viewpoint synthesis is a very resource hungry process and there is the need to find the best tradeoff between the quality of the synthetized view, which is related to the number of the delivered camera streams, and the processing time of the algorithm. The required camera streams may change continuously due to the free navigation of viewpoint, hence effective delivery schemes are required to avoid starvation of the viewpoint synthesizer algorithm and keep the network traffic as low as possible. Moreover, packet losses due to congestion and the increased latency can disturb the user experience. In order to support more multi-view videos in IP networks, a simple approach is to minimize the bandwidth consumption by transmitting only the minimal number of camera views

Optimized Camera Handover Scheme in Free Viewpoint Video Streaming

Free-viewpoint video (FVV) is a promising approach that allows users to control their viewpoint and generate virtual views from any desired perspective. The individual user viewpoints are synthetized from two or more camera streams and correspondent depth sequences. In case of continuous viewpoint changes, the camera inputs of the view synthesis process must be changed in a seamless way, in order to avoid the starvation of the viewpoint synthesizer algorithm. Starvation occurs when the desired user viewpoint cannot be synthetized with the currently streamed camera views, thus the FVV playout interrupts. In this paper we proposed three camera handover schemes (TCC, MA, SA) based on viewpoint prediction in order to minimize the probability of playout stalls and find the tradeoff between the image quality and the camera handover frequency. Our simulation results show that the introduced camera switching methods can reduce the handover frequency with more than 40%, hence the viewpoint synthesis starvation and the playout interruption can be minimized. By providing seamless viewpoint changes, the quality of experience can be significantly improved, making the new FVV service more attractive in the future

Reinforcement Learning based Gateway Selection in VANETs

In vehicular ad hoc networks (VANETs), providing the Internet has become an urgent necessity, where mobile gateways are used to ensure network connection to all customer vehicles in the network. However, the highly dynamic topology and bandwidth limitations of the network represent a significant issue in the gateway selection process. Two objectives are defined to overcome these challenges. The first objective aims to maximize the number of vehicles connected to the Internet by finding a suitable gateway for them depending on the connection lifetime. The second objective seeks to minimize the number of connected vehicles to the same gateway to overcome the limitation of gateways\u27 bandwidth and distribute the load in the network. For this purpose, A gateway discovery system assisted by the vehicular cloud is implemented to find a fair trade-off between the two conflicting objectives. Proximal Policy Optimization, a well-known reinforcement learning strategy, is used to define and train the agent. The trained agent was evaluated and compared with other multi-objective optimization methods under different conditions. The obtained results show that the proposed algorithm has better performance in terms of the number of connected vehicles, load distribution over the mobile gateways, link connectivity duration, and execution time

Optimization of Wi-Fi Access Point Placement for Indoor Localization

The popularity of location based applications is undiminished today. They require accurate location information which is a challenging issue in indoor environments. Wireless technologies can help derive indoor positioning data. Especially, the Wi-Fi technology is a promising candidate due to the existing and almost ubiquitous Wi-Fi infrastructure. The already deployed Wi-Fi devices can also serve as reference points for localization eliminating the cost of setting up a dedicated system. However, the primary purpose of these Wi-Fi systems is data communication and not providing location services. Thus their positioning accuracy might be insufficient. This accuracy can be increased by carefully placing the Wi-Fi access points to cover the given territory properly. In this paper, our contribution is a method based on simulated annealing, what we propose to find the optimal number and placement of Wi-Fi access points with regard to indoor positioning. We investigate its performance in a real environment scenario via simulations

Parking Assistance System for Indoor Environment

In the last decade, indoor location based applications, such as touristic guide in museums or shopping guidance in supermarkets, have been developed rapidly requiring suitable and accurate indoor positioning. However, location sensing in indoor environments is a challenging task and an intensively researched topic. Fortunately, wireless technologies can help us derive location information. In this paper, we propose and introduce our positioning and navigation system for indoor parking garage environment, called iParking, which has been under development. The iParking system collects real-time parking lot occupancy data, and tracks and navigates vehicles entering the parking garage to a preselected, e.g., the closest to the favorite shop, free parking lot. The driver’s smartphone is used as the navigation interface. The system is built on a background Wi-Fi infrastructure making the deployment and maintenance economical. Currently, we have been implementing a prototype of our iParking system in a parking garage of a shopping mall for demonstration purposes

Intelligent Indoor Parking

Nowadays positioning based navigation is an integrated part of our everyday’s routine. Hence, it is hard to succeed without a GPS based navigation system in a bigger city today. However, indoor positioning and navigation are still in their infancy, although these services would be desirable in many areas. One obvious application domain is vehicle navigation in a parking garage. The use of an indoor vehicle navigation system is convenient for the drivers, decreases the unnecessary circling in the garage and reduces air pollution. In this paper, we introduce our iParking indoor positioning and navigation system which has been under development. Our system monitors the occupancy of the parking lots, and with the aid of a Wi-Fi based background wireless infrastructure tracks the position of the vehicle entering the parking garage and navigates the driver to an appropriate free parking lot. Lot selection is handled at the entry point of the garage based on simple preferences, eg., the closest disabled parking space. The navigation interface is the driver’s smartphone. Currently, we have been implementing a prototype of our iParking system in a parking garage of a shopping mall for demonstration purposes

On the Placement of Wi-Fi Access Points for Indoor Localization

Nowadays, the more and more popular location based applications require accurate position information even in indoor environments. Wireless technologies can be used to derive positioning data. Especially, the Wi-Fi technology is popular for indoor localization because the existing and almost ubiquitous worldwide Wi-Fi infrastructure can be reused lowering the expenses. However, the primary purpose of these Wi-Fi systems is different from being used for positioning services, thus the accuracy they provide might be low. This accuracy can be increased by carefully placing the Wi-Fi access points to cover the given territory appropriately. In this paper, we propose a simulated annealing based method to find, in a given area, the optimal number and placement of Wi-Fi access points to be used for indoor positioning. We investigate the performance of our method via simulations

Analyzing energy efficiency of sensor networks deployed on the surface of a Solar System Body

Following the exploration of Moon, the next step could be the exploration of Mars, since many man-made devices already sent measurement results from our closest planet. There is an interesting tendency that the private funding space researches are becoming more and more substantial next to state sponsored space programs. In the process of mapping a distant planet, cost efficiency is high priority since the available resources are limited. A cost-efficient method is using sensor networks to explore, which can be done on a lower budget compared to “single-probe” missions [1]. Human intervention is often not possible due to the great distances. Therefore, the usage of sensor networks can partly be a solution to the arising problems, since losing connection with the home base on Earth does not hinder the measurements [2]. Another advantage is that the failure of a device does not put the mission at jeopardy [3]. In our work, we assumed such a sensor network, which we examined from different points of view. Real Martian topographical data was used to create a Digital Elevation Model on which we studied different sensor movement algorithms. We analyzed the communication between sensors from the energy-efficiency aspect and we established an energy model to estimate the resource consumption of the sensors. A simulation program has been developed to examine our sensor network. In this simulation, we compared the efficiency of the algorithms and we investigated how the energy level of the sensors affect the time required to cover the measurement area